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Introduction 


This study sought to estimate the potential 
benefits of using ACT® WorkKeys® 
Curriculum to improve foundational 
workplace skills and subsequent 
performance on the ACT® WorkKeys® 
Applied Math, Workplace Documents, and 
Graphic Literacy assessments. Students at 
Nevada State High School (NSHS) took the 
WorkKeys Assessments before and after 
using ACT WorkKeys Curriculum. Using 
data from the supervisor dashboard, 
students were classified as engaged or 
unengaged in lessons, practice questions, 
and quizzes included in WorkKeys 
Curriculum. Overall, students who engaged 
meaningfully with WorkKeys Curriculum 
were expected to have greater score gains 
and were more likely to increase their 
WorkKeys level scores. In particular, 
students who engaged in Applied Math 
practice questions had significantly greater 
scale score gains than unengaged students, 
and students who exhibited overall 
engagement in the Applied Math and 
Workplace Documents modules were more 
likely to increase their corresponding level 
scores. Engagement in WorkKeys 
Curriculum was related to performance 
improvements for Graphic Literacy but not 
to a statistically significant degree. 


Additional analyses detected significant 
relationships between WorkKeys scores 
and grade point average at NSHS. In all, 
results suggest that engagement with 
WorkKeys Curriculum helps improve 
WorkKeys scores and that WorkKeys is a 
useful predictor of academic success in a 
dual-enrollment, college-into-workforce 
program. 


Nevada State High School 


NSHS is a dual-enrollment system of 
schools in Nevada for 11th and 12th 
graders that has been operating since 2004. 
The students at NSHS attend college 
classes with college professors for a real 
college experience. NSHS provides classes 
to help monitor and support students in their 
early college experience. The curriculum 
focuses on readiness for the transition from 
college to careers with real world 
applications. This involves working with 
students to help develop workplace skills 
and career direction for their post-high 
school college planning. NSHS has been 
using WorkKeys Assessments since the 
2008-2009 school year as a way to 
compare student performance with the 
foundational workplace skill levels required 
for jobs nationwide. Moreover, WorkKeys 
results are meaningful for students after 


ACT 


00006 


ACT.org/research 
© 2020 by ACT, Inc. All rights reserved. | R1793 


ACT Research & Policy | Technical Brief | January 2020 


high school because prospective employers 
may use WorkKeys for hiring purposes, and 
most students earn a nationally recognized 
certification—the ACT National Career 
Readiness Certificate. 


Background 


ACT WorkKeys is an assessment of 
foundational workplace skills used by 
employers and workforce developers to 
select job candidates and trainees. 
WorkKeys Curriculum is designed to help 
users improve the skills measured by the 
WorkKeys Applied Math, Workplace 
Documents, and Graphic Literacy 
assessments. For each skillset, the 
curriculum materials are divided by levels 
(3, 4, 5, 6, and 7) corresponding to levels of 
knowledge and skills that are required for 
different jobs or training programs." 
Curriculum users have the option of taking a 
placement test to estimate their initial skill 
level. For each WorkKeys level, the 
curriculum provides instruction, practice 
items, and a summative quiz to evaluate 
mastery. 


All first-year students at NSHS (11th or 12th 
grade) take the WorkKeys Applied Math, 
Workplace Documents, and Graphic 
Literacy assessments. Students who do not 
achieve a level score of 5 or higher on a 
given assessment are assigned to the 
corresponding modules of WorkKeys 
Curriculum. Students subsequently take 
certain WorkKeys assessments again to 
attempt earning a higher score. This pre- 
test/post-test design provides a unique 
opportunity to examine the effectiveness of 
WorkKeys Curriculum for improving 
WorkKeys scores. 


A preliminary analysis of 2017-2018 data 
from NSHS indicated that students who 


engaged with the Applied Math and 
Workplace Documents modules of 
WorkKeys Curriculum increased their 
scores approximately 1 point more, on 
average, than students who did not engage 
with the curriculum.* Unfortunately, students 
in 2017-2018 were not assigned to the 
practice questions, which might be expected 
to contribute significantly to the 
effectiveness of WorkKeys Curriculum. In 
2018-2019, NSHS students were assigned 
all components of WorkKeys Curriculum, 
which made it possible to evaluate the full 
implementation of the curriculum. This 
report presents results from analyses of the 
2018-2019 data. 


Data 


Data from several standardized 
assessments were analyzed for this study. 
WorkKeys scores are reported on a 65—90 
scale, and those scale scores are 
transformed to level scores with a maximum 
of 7. Level scores are commonly used to 
support hiring decisions and admissions to 
workforce training programs. Examinees 
who achieve certain level scores on the 
Applied Math, Workplace Documents, and 
Graphic Literacy assessments are awarded 
an ACT® National Career Readiness 
Certificate® (NCRC®).° Specifically, 
examinees earn a Bronze NCRC by scoring 
level 3 or higher on all assessments, Silver 
by scoring level 4 or higher, Gold by scoring 
level 5 or higher, and Platinum by scoring 
level 6 or higher. 


The data also included scores from the 
ACT® assessment, which is primarily used 
for college admissions. For the ACT, each 
examinee completes four subject-area tests 
(English, reading, math, and science), and 
scores are reported on a 1-36 scale. The 
average of the subject-area scores is the 
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ACT Composite score. Students at NSHS 
also took ACT Engage®, a measure of 
social and emotional learning competencies 
associated with academic success. The full 
instrument includes 108 items divided 
between 10 subscales. ACT Engage school 
and district score reports include two 
indices: an academic success index 
reflecting the probability of earning a 2.0 or 
higher GPA, and a retention index reflecting 
the probability of completing high school; 
both were analyzed in this study. 


NSHS provided ACT a de-identified data file 
including records for students in 2018- 
2019. This file included demographics, 
grade point average, math courses taken, 
WorkKeys scores, and ACT Engage scores. 
Using a unique identification number, the 
NSHS data were merged with records 
exported from WorkKeys Curriculum. This 
included percentage of instructional lessons 
viewed, total time spent on lessons, number 
of practice questions completed, average 
practice question response time, 
percentage correct on practice questions, 
number of quizzes completed, average quiz 
question response time, and percentage 
correct on quizzes (overall and for each 
level). 


Sample 


The overall sample included 248 NSHS 
students, 70.2% of which were female. In 
terms of race/ethnicity, the sample was 
6.0% Asian, 11.7% Black, 40.7% Hispanic, 
1.6% Pacific Islander, 29.0% White, and 
10.9% multi-racial/ethnic. Four percent of 
students were classified as English 
language learners during 2018-2019; an 
additional 17.3% had been classified as 
English language learners in prior years. As 
an indicator of socioeconomic status, 47.6% 


of the sample was eligible for free or 
reduced-price lunch. Analyses were 
conducted on smaller groups of students 
who retested on Applied Math (186), 
Workplace Documents (159), or Graphic 
Literacy (63). The distributions of 
demographic variables were similar in each 
analysis, so only overall sample 
demographics are reported here. 


Results 


Descriptive Statistics 


Table 1 provides descriptive statistics for 
WorkKeys, the ACT, Engage, and GPA for 
the different (but overlapping) samples 
analyzed for this study. Pre-test WorkKeys 
scores reflect the earliest test date during 
the 2018-2019 school year, and post-test 
scores reflect that latest test date. The 
average maximum WorkKeys scores were 
slightly higher than the average post-test 
scores because students did not always 
achieve their highest scores on the last test 
date. Comparing ACT Composite score, the 
Engage indices, and GPA between samples 
indicates that the Graphic Literacy sample 
had relatively low average academic 
achievement and social-emotional learning 
competencies. 


Table 2 shows the correlations between 
variables analyzed in this study. ACT 
Composite score had moderate and 
statistically significant correlations with all 
three maximum WorkKeys scores. That is, 
students who earned higher ACT 
Composite scores tended to earn higher 
WorkKeys scores. Of the WorkKeys 
Assessments, social-emotional learning 
competencies were most strongly related to 
Graphic Literacy scores. Nearly all 
WorkKeys, ACT, and Engage scores 
correlated significantly with GPA. 
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Table 1. Average Assessment Scores and Grade Point Averages 


Applied Math = Documents Literacy 

(n = 159) (n = 63) 
WorkKeys Pre-Test 76.2 77.9 75.6 
WorkKeys Post-Test 78.1 79.8 78.0 
WorkKeys Maximum 78.6 80.2 78.2 
ACT Composite 17.8 17.8 16.6 
Engage Academic Success A5 45 39 
Engage Retention A1 A1 .36 
Incoming GPA 3.13 3.19 2.94 
Outgoing GPA 3.20 3.28 3.02 
WorkKeys Pre-Test 76.2 77.9 75.6 
WorkKeys Post-Test 78.1 79.8 78.0 
WorkKeys Maximum 78.6 80.2 78.2 
ACT Composite 17.8 17.8 16.6 


Table 2. Correlations with Outcome Variables 


Workplace (etry eal (es 


Applied Math Documents Literacy Cumulative 

MERE Maximum Maximum Lee 
Applied Math Pre-Test 65" 28" .09 29" 
Applied Math Post-Test 93" 43" 28° 29" 
Applied Math Maximum 43™" 22 32" 
Workplace Documents Pre-Test as 467 AS of 
Workplace Documents Post-Test 46™ 92" 25 2T 
Workplace Documents Maximum 437 22 2 
Graphic Literacy Pre-Test au: 14 19 AG 
Graphic Literacy Post-Test .24 .23 96" 20. 
Graphic Literacy Maximum Ze 22 2T 
ACT Composite 56" 53°" 40” 46" 
Engage Academic Success 16 07 or s2” 
Engage Retention 16 05 oo 46™ 


"p< .05,™p<.01, p< .001 


WorkKeys Curriculum Engagement practice and quiz questions of two seconds 
or less. On one hand, this observation is 
disappointing because students missed 
chances to improve their foundational 
workplace skills. For research purposes, 
however, this provided an opportunity to 
examine the effectiveness of WorkKeys 


Usage data from WorkKeys Curriculum 
made it apparent that some students 
engaged meaningfully with the curriculum 
and some did not. For example, numerous 
students had average response times for 
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Curriculum by comparing students who did 
and did not engage. 


Engagement was inferred using data from 
the WorkKeys Curriculum dashboard 
including number of lessons, practice 
questions, and quizzes; time spent on 
lessons, practice questions, and quizzes; 
and percentage correct scores on practice 
questions and quizzes. Cut points were 
established through a combination of 
empirical and rational means. This generally 
involved examining the distribution of a 
variable, identifying a value that would 
isolate the group of apparently disengaged 
students, and comparing that value with 
expectations considering the amount of 
curricular materials and the amount of time 
it would take to read or respond 
conscientiously. For example, there was a 
sizeable group of students who spent less 
than 40 minutes reviewing the lessons. 
Considering that there were between 21 
and 30 lessons per module, and each 
lesson entailed several pages of text and 
example questions, students who spent less 
than 40 minutes on the lessons could not 
have reviewed many of them thoroughly. 
The cut point for question response time (15 
seconds) was based on the distribution of 
response times and prior research on 
responding behavior indicative of rapid 
guessing. The percentage correct cut point 
(55%) was chosen to be higher than the 
percentage expected from random guessing 
(25%) and indicative of effort to answer 
questions correctly, yet well below full 
mastery, which was not necessarily 
expected of students. Cut points for 
percentage of completed practice questions 
(30%) and number of quizzes (three out of 
five) were selected based on distributions 
and what would be considered low to 
moderate effort. 


The goal was to identify students who were 
likely not engaged. Setting the cut points 
higher might have identified students who 
were more engaged, but exploratory 
analyses indicated that results were not 
sensitive to higher cut points. In most 
analysis, variables were combined because 
individual variables were unreliable 
indicators of engagement. For example, 
some students completed every practice 
question but got only 30% correct—an 
indication of random guessing. 


As described above, engagement was first 
inferred by classifying students as engaged 
in the lessons (i.e., reading instructional text 
and reviewing example questions) 
according to the amount of time spent on 
the lessons (40+ minutes). The percentage 
of lessons viewed was not considered 
because students could easily demonstrate 
substantial “progress” through the lessons 
in a very short time. As shown in Table 3, 
90% of students engaged in the Applied 
Math lessons, 77% in the Workplace 
Documents lessons, and 76% in the 
Graphic Literacy lessons. 


Students were classified as engaging in the 
practice questions if they spent an average 
of 15 or more seconds responding to each 
practice question, completed at least 30% of 
the available practice questions, and got at 
least 55% correct on the practice questions. 
The percentage engaging on the Applied 
Math practice questions (61%) was greater 
than the percentage on the Workplace 
Documents (36%) or Graphic Literacy 
(35%) practice questions (Table 3). The 
main drivers of lack of engagement were 
short response times and low percentages 
correct. 


WorkKeys Curriculum is divided into units 
based on WorkKeys levels 3 through 7, and 
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each unit concludes with a quiz to assess quizzes was particularly low for Graphic 
mastery of associated skills. Students were Literacy (14%; Table 3). As with the practice 
classified as engaged on the quizzes if they questions, students were most likely to be 
spent an average of 15 or more seconds flagged for disengagement due to short 
responding to quiz questions, completed response times and low percentages 

three or more quizzes (out of five), and got correct. 


at least 55% correct on the quiz questions. 
The percentage of student engaged in 


Table 3. Average Assessment Scores and Grade Point Averages 


Voy eI t-ve| Workplace fet e-J el alles 
Math Documents Literacy 
Mean Median Mean Median Mean WY CeXo TFTA) 
Average Progress 84% 100% 82% 100% 81% 100% 
Total Time (min.) 229 208 117 101 122 93 
% Practice Questions Completed 85% 31% 78% 100% 100% 0% 
Mean Practice Response Time (sec.) 71 A1 205 27 79 18 
Practice % Correct 72% 74% 66% 68% 67% 71% 
Quizzes Completed 4.2 5.0 4.0 5.0 4.0 5.0 
Mean Quiz Response Time (sec.) 79 35 86 36 1527 20 
Quiz % Correct 70% 78% 67% 75% 67% 77% 
Engaged in Lessons 90% 77% 76% 
Engaged in Practice 61% 36% 35% 
Engaged in Quizzes 48% 40% 14% 
Engaged in All 38% 22% 10% 
Pre-Test/Post-Test Differences distributions’). The average differences 


corresponded to moderate effect sizes. 
On average, students increased their P 


WorkKeys scores from the initial 
administration (pre-test) to the final 
administration (post-test) by an average of 
1.9 to 2.4 points on the 65-90 WorkKeys 
scale (Table 4). Average pre-test to 
maximum score differences ranged from 2.3 
to 2.6 points. All differences were 
statistically significant according to paired- 
sample t-tests. The average differences 
were transformed to effect sizes, which are 
reported in standard deviation units (using 


Figures 1, 2, and 3 illustrate changes in the 
distributions of WorkKeys level scores over 
time for Applied Math, Workplace 
Documents, and Graphic Literacy, 
respectively. In all cases, retesting resulted 
in reductions in the percentages of level 3 
and level 4 scores and increases in the 
percentages of level 5 and level 6 scores. 
Correspondingly, the percentages of 
students with a Bronze or Silver NCRC 

a decreased and the percentage with a Gold 
standard deviations from population NCRC increased (Figure 4; NCRC levels 
are based on maximum scores). 
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Table 4. Comparison of Mean WorkKeys Scale Scores 


Mean Difference Effect Size 
Pre- Pre- 

Assessment A Post ENS 
Applied Math 186 76.2 78.1 78.6 1.9" 24” 0.35 0.44 
Workplace 159 779 798 80.2 1.9°" oe i 0.37 0.45 
Documents 
Graphic ia in 

é 63 75.6 78.0 78.2 2.4 2.6 0.47 0.50 
Literacy 


“p< .05,” p<.01, p< .001 


Figure 1. Applied Math Level Score Distributions 
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80% 


60% 


4 Lo} 
ee 27% 27% 
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Applied Math Level Score 


Figure 2. Workplace Documents Level Score Distributions 


mPre-Test ™Post-Test mMaximum 


100% 
80% 


60% 


40% 28% 28% 


20% 16% 16% 
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Figure 3. Graphic Literacy Level Score Distributions 
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Figure 4. National Career Readiness Certificate Level Distributions 
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WorkKeys Curriculum Effect 


The observed increases in WorkKeys scale 
scores and level scores are possibly 
attributable to use of WorkKeys Curriculum. 
However, students took the post-test an 
average of six months after the pre-test, and 
learning may have occurred through other 
means, such as classroom instruction. The 
following analyses help address this 
limitation by controlling for engagement with 
WorkKeys Curriculum. Assuming that 
students experienced similar learning 
outside of WorkKeys Curriculum (regardless 


of engagement with the curriculum), the 
analysis attempted to disentangle learning 
within and outside WorkKeys Curriculum. 


The first analysis employed linear 
regression—specifically, analysis of 
covariance—to examine the relationship 
between engagement with WorkKeys 
Curriculum and post-test scores while 
controlling for pre-test scores. Results 
indicated whether students who engaged 
with WorkKeys Curriculum earned higher 
post-test scores than unengaged students 
with similar pre-test scores. A total of five 
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models were fit to the data for each 
WorkKeys Assessment (see Appendix 
Table A1). The models included various 
combinations of engagement indicators 
(engaged in lessons, engaged in practice, 
engaged in quizzes, and engaged in all 
three) and the ACT Engage Academic 
Success index. 


Of the engagement variables, engagement 
in practice appeared to have the strongest 
association with post-test scores (Appendix 
Table A1). For Applied Math, students who 
engaged in the practice questions earned 
post-test scores that were 1.9 points higher 
than students with similar pre-test scores 
who did not engage (Model 3). The left 


panel of Figure 5 illustrates this relationship. 


This statistically significant difference 
corresponded to an effect size of 0.35 
standard deviations (a moderate effect). 
The expected effect of engagement in 
practice questions was approximately 0.5 
points for Workplace Documents, but this 
coefficient was not significantly different 
from zero. The models including only an 
overall engagement indicator each 
suggested a positive effect associated with 
engagement (Model 4), but only the 
coefficient for Applied Math (1.1) was 
statistically significant (see Model 3 in 
Appendix Table A1). Smaller sample sizes 
for Workplace Documents and Graphic 
Literacy made it more difficult to detect 
Statistically significant effects. 


The last model included engagement in 
practice questions and the ACT Engage 
Academic Success index. The Academic 
Success and Retention indices were highly 
correlated (r= .91); therefore, the Retention 
index was left out of analyses to avoid 
multicollinearity. Consistent with the 
correlations reported in Table 2, the 
coefficient for Academic Success was 


statistically significant only for the Graphic 
Literacy model (see Model 5 in Appendix 
Table A1). Note that the Academic Success 
index reflects a combination of academic 
discipline, social activity, and a self-reported 
ACT Composite score. In general, harder 
working and more conscientious students 
tend to have better educational outcomes. 
Thus, Academic Success was expected to 
predict WorkKeys scores. 


Next, logistic regression was used to 
examine the relationship between 
engagement in WorkKeys Curriculum and 
the probability of increasing by one level 
score or more (based on maximum 
WorkKeys scores). In this analysis, a 
significant effect indicated that students who 
engaged with WorkKeys Curriculum had 
higher odds of increasing their level scores 
through retesting than unengaged students 
with similar pre-test scores. The general 
patterns in logistic regression results 
(Appendix Table A2) were similar to the 
linear regression results. That is, students 
who engaged in the Applied Math practice 
questions had significantly greater odds of 
increasing their level scores. The right panel 
of Figure 5 reveals that engaged students’ 
chances of increasing their level scores 
were approximately 25% higher than 
unengaged students. The negative slope 
indicates that that students with higher pre- 
test scores were somewhat less likely to 
increase their level scores.°® 


Figure 6 illustrates how overall engagement 
in the Workplace Documents curriculum 
materials related to expected post-test 
scores and the probability of increasing 
one’s level score. The effect of overall 
engagement (0.73) was not significantly 
different from zero in the linear regression, 
but the overall engagement coefficient in 
logistic regression was nearly so (p = .07). 


Applied Math Post-Test 


Workplace Documents Post-Test 
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Throughout the range of Workplace Literacy curriculum were positive and similar 
Documents pre-test scores, students who in magnitude to those for Workplace 
engaged in WorkKeys Curriculum had Documents, but neither was statistically 
greater chances of improving their level significant (Table A2). 


scores (by up to 19%). The estimated 
effects of overall engagement in the Graphic 


Figure 5. Applied Math Linear and Logistic Regression Plots for Engagement in Practice 
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Figure 6. Workplace Documents Linear and Logistic Regression Plots for Overall Engagement 
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Predicting Outgoing GPA 


The main objective of this study was to 
examine the effectiveness of WorkKeys 
Curriculum for improving foundational 
workplace skills as measured by WorkKeys. 
However, data provided by NSHS allowed 
for additional analyses relevant to the 
validity of WorkKeys as a predictor of 
success in dual-enrollment programs like 
NSHS. Data were analyzed from a total of 
217 students who took WorkKeys during the 
2018-2019 academic year. For this 
analysis, it was not necessary to restrict the 
sample to students who used WorkKeys 
Curriculum. Included students had complete 
records including GPA, WorkKeys, ACT, 
and Engage. This analysis used the earliest 
WorkKeys scores (pre-test) as predictors of 
cumulative GPA. As indicated by the 
correlations in Table 2, the single best 
predictor of cumulative GPA was incoming 
GPA, which should be expected since both 
are calculated using grades from many of 
the same courses. Indeed, incoming GPA 
accounted for 84.5% of the variance in 
cumulative GPA, and no other variables 
added incrementally to the prediction of 
cumulative GPA. For that reason, incoming 
GPA was held out of predictive models for 
cumulative GPA. 


As shown in Table 5, scores from all three 
WorkKeys Assessments were statistically 
significant predictors of cumulative GPA 
(Model 1). However, their coefficients were 
no longer significant wnen ACT Composite 


11 


score was added to the model (Model 2). 
This finding is possibly explained by the 
correlations among the WorkKeys scores 
(i.e., multicollinearity resulted in imprecise 
model parameter estimates). Indeed, in 
subsequent analyses, Graphic Literacy 
(alone) provided significant incremental 
improvement to the prediction of cumulative 
GPA, but Applied Math and Workplace 
Documents did not. One possible 
explanation is that Graphic Literacy 
measures unique skills that are not fully 
captured by other predictors. ACT 
Composite includes math and reading 
performance, so it may provide information 
that overlaps with Applied Math and 
Workplace Documents. 


Model 3 included Graphic Literacy, ACT 
Composite, and the Engage Academic 
Success index. The Academic Success 
index incorporates self-reported ACT 
Composite score, so it overlaps with ACT 
Composite to some extent. Yet, including 
the Academic Success index as a predictor 
nearly doubled the variance in cumulative 
GPA accounted for by the model (Graphic 
Literacy accounted for only a small 
percentage of the variance in Model 3). 
Thus, the combination of standardized 
achievement tests and a measure of social- 
emotional learning competencies provided 
useful prediction of cumulative GPA for 
NSHS students, but they did not improve 
upon the predictive accuracy provided by 
incoming GPA. 
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Table 5. Linear Regression Model Coefficients for Predicting Outgoing GPA 


Model 1 ore (=) 4 NM Coke (=) Is) 
Intercept 5.513" -2.499 -1.115 
Applied Math 0.032" 0.014 
Workplace 0.040° 0.021 
Documents 
Graphic Literacy 0.040° 0.020 0.034" 
ACT Composite 0.078" 0.065" 
Academic 0.011" 
Success 
R2 146 217 A405 


*p < .05, “p< .01, p< .001 


Summary 


This study estimated how engaging with WorkKeys Curriculum related to changes in 
performance on the WorkKeys Applied Math, Workplace Documents, and Graphic Literacy 
assessments for students at NSHS. Students were identified as “engaged” according to the time 
spent on lessons, number of practice questions and quizzes completed, time spent answering 
practice and quiz questions, and percentage correct on practice and quiz questions. In statistical 
analyses, the strongest evidence that engagement in WorkKeys Curriculum translates to higher 
WorkKeys scores was observed for Applied Math. Specifically, students who engaged in 
Applied Math practice questions were expected to increase their scores by approximately 2 
points more than unengaged students, and they were significantly more likely to improve their 
level scores. There was also statistical evidence that students who engaged in all aspects of the 
Workplace Documents curriculum were more likely to improve their level scores. Thus, fairly 
brief engagement with WorkKeys Curriculum (a few hours) was associated with greater 
improvements in WorkKeys performance. 


A substantial percentage of students did not apparently engage with WorkKeys Curriculum as 
intended—in particular for the Workplace Documents and Graphic Literacy modules. The 
engagement cutoff values were selected to reflect minimal engagement. In future research, 
different cutoff values could be applied for different WorkKeys Curriculum modules, or multiple 
cutoffs might differentiate between levels of engagement. For that to work, however, more 
students would have to engage at higher levels. The ability to monitor student activity in 
WorkKeys Curriculum was recently improved, so NSHS instructors may be able to ensure that 
students complete their assignments conscientiously in the future. Supplementary analyses 
compared the relative value of predictors of cumulative GPA at NSHS. The WorkKeys 
Assessments all correlated significantly with cumulative GPA, though regression analyses 
revealed ACT Composite and ACT Engage scores to be relatively strong predictors of 
cumulative GPA. In all, this study provided evidence supporting the use of WorkKeys 
Curriculum for improving WorkKeys scores and the value of the skills measured by Applied 
Math, Workplace Documents, and Graphic Literacy for success in dual-enrollment, college-into- 
workforce programs like Nevada State High School. 
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smaller gains, and students who perform particularly poorly initially (relative to their “true” 
abilities) are expected to exhibit greater gains. 
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Appendix 


Table A1. Linear Regression Model Coefficients for Predicting WorkKeys Post-Test Score 


LN CoXe (=9 Bs) 
Model 3 (Practice & 
I oye C=) 4 (Practice Model 4 yXer-Ye(-Vnaliey 
WETIELe) (= i Coke (=1 i (All Three) Only) COT 11)) Success) 
Intercept 15.099 17.009" 15.965" 18.344" 16.226" 
Pre-Test 0.827" 0.794" 0.801" 0.779" 0.787" 
Engaged in Lessons -0.732 
Applied Engaged in Practice 1.939" 1.863" 2.405” 
Math Engaged in Quizzes 0.233 
Engaged in All 1.061" 
Academic Success 0.010 
R? .268 334 .330 .287 .380 
Intercept 44.824” 46.690" 46.710 47.080" 44.951" 
Pre-Test 0.449" 0.426™ 0.423™ 0.418" 0.445" 
Engaged in Lessons -0.360 
Workplace Engaged in Practice 0.519 0.450 0.418 
Documents Engaged in Quizzes -0.011 
Engaged in All 0.732 
Academic Success 0.002 
R? .105 .114 sal .117 114 
Intercept 66.727 68.142" 66.654" 70.632" 74.976" 
Pre-Test 0.149 0.137 0.150 0.096 0.021 
Engaged in Lessons -0.845 
Graphic Engaged in Practice 0.388 0.083 0.389 
Literacy Engaged in Quizzes 0.015 
Engaged in All 0.921 
Academic Success 0.036" 
R? 014 .039 015 029 .176 


“p<.05," p<.01, "p< .001 
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Table A2. Logistic Regression Model Coefficients for Predicting Probability of Increasing WorkKeys Level 
Score 


i KoXe f=) Ie) 
WM CoXe (= is) (Practice & 
LM Coke (21 7 (Practice Model 4 yXer-Ye(-1nal ley 
AYE-Tar-10) (3 I Coke [1 i (All Three) Only) (ON Z21¢-11)) Success) 
Intercept 0.943 1.648 1.549 2.871 2.849 
Pre-Test -0.010 -0.030 -0.026 -0.038 -0.051 
Engaged in Lessons 0.293 
Applied Engaged in Practice 0.982” 1.066 0.000 1.472 
Math Engaged in Quizzes 0.070 
Engaged in All 0.626" 
Academic Success 0.009 
McFadden's R? .000 .049 .047 015 213 
Intercept 10.891 13.512 13.029 13.342" 9.633 
Pre-Test -0.137 -0.169 -0.167 -0.171° -0.121 
Engaged in Lessons -0.447 
Workplace Engaged in Practice 0.518 0.482 0.440 
Documents Engaged in Quizzes 0.116 
Engaged in All 0.757 
Academic Success -0.001 
McFadden's R? 013 .027 022 029 .130 
Intercept 26.368 21.468 26.289 28.825 37.423" 
Pre-Test -0.336 -0.273 -0.335 -0.369 -0.497° 
Engaged in Lessons 0.289 
Graphic Engaged in Practice 0.127 0.075 0.447 
Literacy Engaged in Quizzes -0.604 
Engaged in All 0.404 
Academic Success 0.029" 
McFadden's R? 042 049 042 044 .215 


“p< .05,” p<.01,"™ p< .001 
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